Imaging apparatus, imaging method and imaging program

ABSTRACT

An imaging apparatus is provided that includes: a video input unit configured to take an image of an object to generate an image signal of the object; a video signal processor configured to generate a taken image of the object on a basis of the image signal; and a controller configured to: detect motion information of the object on the basis of the image signal; cause the video input unit to take an image of the object on a basis of the motion information multiple times so as to differentiate an exposure amount thereof; and cause the video signal processor to generate an HDR synthetic image of the object on the basis of a plurality of image signals whose exposure amounts are different from each other.

TECHNICAL FIELD

The present invention relates to an imaging apparatus, an imagingcontrol method, and an imaging program, and particularly, the presentinvention relates to an imaging apparatus, an imaging control method andan imaging program configured to synthesize images obtained by taking animage while changing exposure amounts of a plurality of cameras and togenerate an HDR (High Dynamic Range) synthetic image.

BACKGROUND ART

The spread of a digital camera is remarkable. The digital cameraconverts an object image into an electric signal by an electronic devicesuch as a CCD (Charge Coupled Device), for example, and stores theconverted electric signal in a memory. Further, recently, an informationterminal device on which a digital camera is mounted, such as a cellularphone, a smartphone, or a tablet terminal is also spreaded widely.

A dynamic range of an imaging device used in a digital camera or thelike is very narrow compared with a film. Therefore, so-calledunderexposure (or black spot) or overexposure (or white spot) occursdepending upon an imaging condition, whereby image quality is extremelydeteriorated. In order to solve such a defect, an HDR synthesizefunction gains attention. In the HDR synthesize function, a plurality ofimages is taken while changing exposure, dark portions are extractedfrom an image obtained by lengthening exposure and bright portions areextracted from an image obtained by shortening exposure when theseimages are to be synthesized, whereby a broad dynamic range can beacquired while suppressing overexposure and underexposure.

For example, Patent document 1 discloses a video generating apparatusthat has two imaging devices in which one with high resolution takes animage with a small exposure amount and the other with low resolutiontakes an image with a large exposure amount for a short exposure time,whereby an HDR synthetic image is created while reducing noise due tocamera shake or the like.

RELATED ART DOCUMENTS Patent Documents

Patent document 1: Japanese Patent Application Publication No.2007-336561

SUMMARY OF THE INVENTION

However, a technique disclosed in Patent document 1 aims at reduction ofnoise such as camera shake by shortening the exposure time when an imagewith a large exposure amount is to be taken. However, Patent document 1does not consider an HDR synthesizing method to acquire more suitableimage quality based on imaging environment such as motion of an object,brightness, or camera shake. Further, an effective utilization method ofthe two imaging devices is also not mentioned therein.

It is thus an object of the present invention to provide an imagingapparatus, an imaging control method and a program configured togenerate a high-quality HDR synthetic image compatible with imagingenvironment.

Means for Solving the Problem

An outline of representative invention of the present inventiondisclosed in the present application will briefly be explained asfollows.

An imaging apparatus according to a representative embodiment of thepresent invention includes: a video input unit configured to take animage of an object to generate an image signal of the object; a videosignal processor configured to generate a taken image of the object on abasis of the image signal; and a controller configured to: detect motioninformation of the object on the basis of the image signal; cause thevideo input unit to take an image of the object on a basis of the motioninformation multiple times so as to differentiate an exposure amountthereof; and cause the video signal processor to generate an HDRsynthetic image of the object on the basis of a plurality of imagesignals whose exposure amounts are different from each other.

Effects of the Invention

Effects obtained by representative invention of the present inventiondisclosed in the present application will briefly be explained asfollows.

Namely, according to a representative embodiment of the presentinvention, it becomes possible to provide an imaging apparatus, animaging control method and a program to generate a high-quality HDRsynthetic image compatible with imaging environment.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 is an external view illustrating one example of an imagingapparatus 100 according to a first embodiment of the present invention;

FIG. 2 is a block diagram illustrating one example of a configuration ofthe imaging apparatus according to the first embodiment of the presentinvention;

FIG. 3 is a block diagram illustrating one example of a configuration ofthe imaging apparatus according to the first embodiment of the presentinvention;

FIG. 4 is a block diagram illustrating one example of a configuration ofa video signal processor and a video input unit according to the firstembodiment of the present invention;

FIG. 5 is a flowchart related to an imaging method according to thefirst embodiment of the present invention;

FIG. 6 is a flowchart illustrating one example of processes at a motioninformation detecting step, an imaging step, a taken image generatingstep and the like according to the first embodiment of the presentinvention; FIG. 7 is a flowchart illustrating one example of the motioninformation detecting step, the imaging step, the taken image generatingstep and the like according to the first embodiment of the presentinvention;

FIG. 8 is a view illustrating a timing chart of the imaging according tothe first embodiment of the present invention;

FIG. 9 is a view schematically illustrating an HDR synthesizing processaccording to the first embodiment of the present invention;

FIG. 10 is a view schematically illustrating the HDR synthesizingprocess according to the first embodiment of the present invention;

FIG. 11 is a view schematically illustrating the HDR synthesizingprocess according to the first embodiment of the present invention;

FIG. 12 is a block diagram illustrating one example of a configurationof the imaging apparatus according to a second embodiment of the presentinvention;

FIG. 13 is a block diagram illustrating one example of a configurationof the imaging apparatus according to the second embodiment of thepresent invention;

FIG. 14 is a flowchart related to an imaging method according to thesecond embodiment of the present invention;

FIG. 15 is a flowchart related to the imaging method according to thesecond embodiment of the present invention;

FIG. 16 is a view schematically illustrating an HDR synthesizing processaccording to the second embodiment of the present invention;

FIG. 17 is a view schematically illustrating the HDR synthesizingprocess according to the second embodiment of the present invention; and

FIG. 18 is a view schematically illustrating the HDR synthesizingprocess according to the second embodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, examples of embodiments according to the present inventionwill be described with reference to the drawings. Note that theembodiments that will be explained below are one example for realizingthe present invention, are modified or changed appropriately inaccordance with a configuration of an apparatus to which the presentinvention is applied and/or various conditions, and the presentinvention is not limited to the embodiments described below. Further, apart of each of the embodiments (will be described later) may becombined and configured appropriately.

First Embodiment

FIG. 1 is a view illustrating one example of appearance of an imagingapparatus according to the first embodiment of the present invention.FIG. 1(A) is a plan view of an imaging apparatus 100, and FIG. 1(B) is arear view of the imaging apparatus 100. FIG. 2 is a block diagramillustrating one example of a configuration of the imaging apparatus 100according to the first embodiment of the present invention. FIG. 3 is ablock diagram illustrating one example of a software configuration ofthe imaging apparatus 100 according to the first embodiment of thepresent invention.

FIG. 1 illustrates the imaging apparatus mounted on a smartphone.However, in addition to this, the imaging apparatus 100 may be mountedon a cellular phone, PDA (Personal Digital Assistants) such as a tabletterminal, or an information processing apparatus such as a notebook PC(Personal Computer), for example. Further, the imaging apparatus 100 maybe composed of a digital still camera.

As illustrated in FIG. 1 and FIG. 2, the imaging apparatus 100 includesa main controller (controller) 101, a system bus 102, a memory 104, thestorage 110, a video input unit 120, an audio processor 130, anoperating unit 140, a communication processor 150, a sensor unit 160, anextension interface unit 170, a display 121, and the like.

The system bus 102 is a data communication path through which respectivecomponents of the imaging apparatus 100 are connected to each other. Anoutput and an input of data are executed via system bus 102 between themain controller 101 and each of the components in the imaging apparatus100.

The storage 110 is composed of a nonvolatile memory such as a flash ROM(Read Only Memory), an SSD (Solid State Drive), an HDD (Hard DiscDrive), for example. Information is stored in the storage 110 even in astate where electric power is not supplied to the imaging apparatus 100.

As illustrated in FIG. 3, the storage 110 includes a basic operationprogram storage area 110 a, a camera function program storage area 110b, an audio recognition function program storage area 110 c, a timinginforming function program storage area 110 d, an other program storagearea 110 e, a various information/data storage area 110 f, and the like,for example. Basic operation programs by which a basic operation of theimaging apparatus 100 is executed is stored in the basic operationprogram storage area 110 a. A camera function program that realizes acamera function is stored in the camera function program storage area110 b. An audio recognition program that realizes an audio recognitionfunction is stored in the audio recognition function program storagearea 110 c. A timing informing program that realizes a timing informingfunction is stored in the timing informing function program storage area110 d. Other programs and the like are stored in the other programstorage area 110 e. Various kinds of information such as operationsetting values of the imaging apparatus 100 or user information arestored in the various information/data storage area 110 f.

Further, the storage 110 stores a taken image that is taken by theimaging apparatus 100, a dynamic image (or a moving image), an HDRsynthetic image (will be described later), or these thumbnail images,for example, in the various information/data storage area 110 f.Further, the storage 110 stores a new application program downloadedfrom an application server on the Internet in the other program storagearea 110 e, for example.

Further, the storage 110 stores therein a first motion threshold valueand a second motion threshold value larger than the first motionthreshold value. The first and second motion threshold values become astandard when motion of an object (will be described later) isdetermined. These motion threshold value are set appropriately byexperiments and the like, for example.

Various kinds of programs stored in each unit of the storage 110 aredeveloped on the memory 104. When the various kinds of programs areexecuted by the main controller 101, various kinds of executing unitsthat realize functions of the respective programs are established on thememory 104. For example, when the basic operation program stored in thebasic operation program storage area 110 a is developed on the memory104, a basic operation executing unit 104 a that realizes a basicoperation of the imaging apparatus 100 is established on the memory 104.Further, when the camera function program stored in the camera functionprogram storage area 110 b is developed on the memory 104, a camerafunction executing unit 104 b that realizes the camera function isestablished on the memory 104. Further, when an audio recognitionfunction program stored in the audio recognition function programstorage area 110 c is developed on the memory 104, an audio recognitionfunction executing unit 104 c that realizes the audio recognitionfunction is established on the memory 104. Further, when a timinginforming function program stored in the timing informing functionprogram storage area 110 d is developed on the memory 104, a timinginforming function executing unit 104 d that realizes the timinginforming function is established on the memory 104. Further, the memory104 has a temporary storage area 104 e in which data is temporarilystored if needed and the like.

Note that each of the components such as the basic operation executingunit 104 a, the camera function executing unit 104 b, the audiorecognition function executing unit 104 c, or the timing informingfunction executing unit 104 d may be composed of hardware having thesimilar function to that of the corresponding element. Further, thememory 104 may be formed integrally with the main controller 101.

As illustrated in FIG. 1(A), the display 121 is provided on a frontsurface 100 a of the imaging apparatus 100 on which a third video inputunit 125 is provided. The display 121 is a display device such as aliquid crystal panel or an organic EL (Electra Luminescence) panel, forexample. The display 121 displays an HDR synthetic image (will bedescribed later) processed by a video signal processor 122, a takenimage before synthesis, a dynamic image, a thumbnail image and the like.

The audio processor 130 is composed of an audio output unit 131, anaudio signal processor 132, and an audio input unit 133. The audiooutput unit 131 is a speaker, for example. As illustrated in FIG. 1(A),the audio output unit 131 is provided at a peripheral part of thedisplay 121 on the front surface 100 a of the imaging apparatus 100. Theaudio output unit 131 emits audio on the basis of an audio signalprocessed by the audio signal processor 132. The audio input unit 133 isa microphone, for example. As illustrated in FIGS. 1(A) and 1(B), theaudio input unit 133 is provided on a side surface of the imagingapparatus 100 and on the opposite side of the audio output unit 131 withrespect to the display 121. The audio input unit 133 receives an inputof audio from a user or the like of the imaging apparatus 100 to convertthe inputted audio into an audio signal. The audio input unit 133outputs the inputted audio signal to the audio signal processor 132.Note that the audio input unit 133 may be formed separately from theimaging apparatus 100. In this case, the audio input unit 133 and theimaging apparatus 100 may be connected to each other by wiredcommunication or wireless communication.

The operating unit 140 is an instruction input unit that executes aninput of an operational instruction to the imaging apparatus 100. In thepresent embodiment, for example, as illustrated in FIGS. 1(A) and 1(B),the operating unit 140 is composed of a touch panel 140 a, an operationkey 140 b, and an operation key 140 c, which are arranged so as to beoverlapped on the display 121. The operation key 140 b is provided at aside surface of the imaging apparatus 100. The operation key 140 c isprovided near the display 121 on the front surface 100 a of the imagingapparatus 100. However, the operating unit 140 is not required toinclude all of these components, and may be provided with any of thesecomponents. Further, the touch panel 140 a may be formed integrally withthe display 121. Further, the operating unit 140 may be composed of akeyboard or the like (not illustrated in the drawings), which isconnected to the extension interface unit 170 (will be described later).Further, the operating unit 140 may be composed of a separateinformation terminal apparatus that is connected thereto via wiredcommunication or wireless communication.

The communication processor 150 is composed of a LAN (Local AreaNetwork) communication unit 151, a mobile telephone networkcommunication unit 152, and a proximity wireless communication unit 153,for example. The LAN communication unit 151 executes transmission andreception of data by wireless communication that is connected via anaccess point for wireless communication of the Internet, for example.The mobile telephone network communication unit 152 is connected to abase station of a mobile phone communication network, and executestelephone communication (call) and transmission and reception of data bywireless communication via the base station. The proximity wirelesscommunication unit 153 executes transmission and reception of data witha reader/writer corresponding to proximity wireless. The LANcommunication unit 151, the mobile telephone network communication unit152, and the proximity wireless communication unit 153 include variouskinds of devices such as an encoder, a decoder, or an antenna (notillustrated in the drawings), for example. Further, the communicationprocessor 150 may include an infrared communication unit that executesinfrared communication and the like.

The sensor unit 160 is a group of sensors to detect a state of theimaging apparatus 100. In the present embodiment, the sensor unit 160 isconstituted by a GPS (Global Positioning System) receiver 161, anacceleration sensor 162, a gyro sensor 163, a geomagnetic sensor 164, alight quantity sensor 165, and a proximity sensor 166, for example. Notethat the sensor unit 160 may include any sensor other than thesesensors.

The GPS receiver 161 receives a GPS signal transmitted from each of aplurality of satellites by using a GPS. The GPS signal received by theGPS receiver 161 is outputted to the main controller 101, for example,and a position of the imaging apparatus 100 is detected on the basis ofthe GPS signal in the main controller 101.

The acceleration sensor 162 measures the magnitude and a direction ofacceleration (for example, gravitational acceleration) that is appliedto the imaging apparatus 100. Measured values of the magnitude and thedirection of the acceleration measured by the acceleration sensor 162are outputted to the main controller 101 as acceleration information,and the acceleration applied to the imaging apparatus 100 is detected onthe basis of the acceleration information in the main controller 101.

The gyro sensor 163 measures angular velocity of the imaging apparatus100, which is generated in a case where the user moves the imagingapparatus 100. The angular velocity measured by the gyro sensor 163 isoutputted to the main controller 101 as angular velocity information,for example. The angular velocity of the imaging apparatus 100 isdetected on the basis of the angular velocity information in the maincontroller 101.

The geomagnetic sensor 164 measures the magnitude and a direction ofearth magnetism that is applied to the imaging apparatus 100. Measuredvalues of the magnitude and the direction of the earth magnetismmeasured by the geomagnetic sensor 164 are outputted to the maincontroller 101 as geomagnetic information. The earth magnetism appliedto the imaging apparatus 100 is detected on the basis of the geomagneticinformation in the main controller 101.

The light quantity sensor 165 measures brightness of the periphery ofthe imaging apparatus 100. The light quantity sensor 165 measures lightquantity of the periphery of the object when an image of the object istaken, for example. A measured value of the light quantity measured bythe light quantity sensor 165 is outputted to the main controller 101 aslight quantity information. The light quantity of the periphery of theimaging apparatus 100 is detected on the basis of the light quantityinformation in the main controller 101.

The proximity sensor 166 measures a proximity status with a thing aroundthe imaging apparatus 100. The proximity sensor 166 measures a distanceand a direction of the thing around the imaging apparatus 100, forexample. A measured value of the proximity status measured by theproximity sensor 166 is outputted to the main controller 101 asproximity status information. The proximity status with the thing aroundthe imaging apparatus 100 is detected on the basis of the proximitystatus information in the main controller 101.

The extension interface unit 170 is a group of interfaces for expandingfunctions of the imaging apparatus 100. The extension interface unit 170includes a video/audio interface 171, a USB (Universal Serial Bus)interface 172, and a memory interface 173, for example.

The video/audio interface 171 is connected to an external video/audiooutput apparatus, and receives an input of a video signal and/or anaudio signal outputted from the video/audio output apparatus. Further,the video/audio interface 171 executes an output or the like of a videosignal and/or an audio signal to the video/audio output apparatus. TheUSB interface 172 is connected to a USB device such as a keyboard, andexecutes an input and an output of information with the USB device. Thememory interface 173 is connected to a memory medium such as a memorycard, and executes an input and an output of data with the memorymedium.

Next, the video input unit 120 and the video signal processor 122 willbe described. FIG. 4 is a block diagram illustrating one example of aconfiguration of the video input unit and the video signal processoraccording to the first embodiment of the present invention. Note thatthe third video input unit 125 is omitted in FIG. 4.

As illustrated in FIG. 2 and FIG. 4, the video input unit 120 includes afirst video input unit 123, a second video input unit 124, the thirdvideo input unit 125, and an exposure controller 126, for example. Asillustrated in FIG. 1(B), the first video input unit 123 and the secondvideo input unit 124 are provided side by side on a back surface 100 bof the imaging apparatus 100, for example. As illustrated in FIG. 1(A),the third video input unit 125 is provided in the vicinity of thedisplay 121 on the front surface 100 a of the imaging apparatus 100.

In FIG. 1(B), the first video input unit 123 and the second video inputunit 124 are provided on the back surface 100 b of the imaging apparatus100. However, they may be provided on the front surface 100 a of theimaging apparatus 100, for example. Further, the first video input unit123 and the second video input unit 124 may be formed integrally witheach other.

As illustrated in FIG. 4, the first video input unit 123 includes afirst imaging optical system I23 a and a first imaging device 123 b, forexample. As illustrated in FIG. 4, the second video input unit 124includes a second imaging optical system 124 a and a second imagingdevice 124 b, for example.

Each of the first imaging optical system 123 a and the second imagingoptical system 124 a is composed of a plurality of lenses, a diaphragm(an iris diaphragm or the like), a mechanical or electronical shutter,and the like, for example. The plurality of lenses focuses incidentlight from the object. Each of the first imaging device 123 b and thesecond imaging device 124 b is composed of a CCD (Charge CoupledDevice), a CMOS (Complementary Metal Oxide Semiconductor) or the like,for example.

Each of the first video input unit 123 and the second video input unit124 takes an image of the object, and generates an image signal of theobject. Specifically, light from the first imaging optical system 123 ais stored in each pixel of the first imaging device 123 b as an electriccharge, whereby input light is converted into an electric signal.Further, light from the second imaging optical system 124 a is alsostored in each pixel of the second imaging device 124 b as an electriccharge, whereby input light is converted into an electric signal. Theelectric signals as an image signal are generated in the first imagingdevice 123 b and the second imaging device 124 b in this manner.

In the present embodiment, the first imaging device 123 b is an imagingdevice with first resolution (high resolution), and the second imagingdevice 124 b is an imaging device with second resolution (lowresolution) that is resolution lower than that of the first imagingdevice 123 b.

The exposure controller 126 defines a diaphragm and a shutter speed ofeach of the first imaging optical system 123 a and the second imagingoptical system 124 a on the basis of an instruction from the maincontroller 101 to control an exposure amount to be inputted to each ofthe first imaging device 123 b and the second imaging device 124 b.

The video signal processor 122 generates a taken image of the object onthe basis of the image signal. As illustrated in FIG. 4, the videosignal processor 122 includes a first image processor 122 a, a secondimage processor 122 b, an image synthesizer 122 c, and an image outputunit 122 d, for example.

The first image processor 122 a is connected to the first imaging device123 b, and converts the electric signal (image signal) outputted fromthe first imaging device 123 b into digital image data with a gradationwidth of predetermined bits. The first image processor 122 a thenexecutes image processing for the digital image data to generateintermediate image data suitable for HDR image synthesis. The firstimage processor 122 a outputs the generated intermediate image data tothe image synthesizer 122 c. Further, the first image processor 122 aoutputs the taken image before synthesis to the storage 110.

The second image processor 122 b is connected to the second imagingdevice 124 b, and converts the electric signal (image signal) outputtedfrom the second imaging device 124 b into digital image data with agradation width of predetermined bits. The second image processor 122 bthen executes image processing for the digital image data to generateintermediate image data suitable for HDR image synthesis. The secondimage processor 122 b outputs the generated intermediate image data tothe image synthesizer 122 c. Further, the second image processor 122 boutputs the taken image before synthesis to the storage 110.

The image synthesizer 122 c synthesizes the two intermediate image datainputted from the first image processor 122 a and the second imageprocessor 122 b to generate an HDR synthetic image as the taken image.The image synthesizer 122 c outputs the generated HDR synthetic image tothe storage 110.

The image output unit 122 d outputs the HDR synthetic image generated bythe image synthesizer 122 c, the taken image before synthesis, thedynamic image, these thumbnail images and the like to the display 121 tocause the display 121 to display them.

As illustrated in FIG. 1(B), a flash unit 129 is provided adjacent tothe first video input unit 123 and the second video input unit 124 onthe backsurface 100 b of the imaging apparatus 100, for example. Theflash unit 129 irradiates flash light to the object when the first videoinput unit 123 and the second video input unit 124 take an image of theobject, for example.

The main controller (controller) 101 is composed of a computer such as amicroprocessor unit. The main controller 101 executes each of theexecuting units 104 a to 104 d formed in the memory 104, whereby thefunction of each program is realized. The main controller 101 thusactuates each of the elements of the imaging apparatus 100.

The main controller 101 detects motion information of the object on thebasis of the image signal generated by the video input unit 120.Further, the main controller 101 causes the video input unit 120 to takean image of the object multiple times on the basis of the motioninformation so as to differentiate an exposure amount thereof.Specifically, the main controller 101 compares the motion informationwith the first motion threshold value and the second motion thresholdvalue, and causes the video input unit 120 to take an image of theobject by switching respective imaging modes (will be described later)on the basis of its result. Further, the main controller 101 generatesexposure information such as exposure amounts regarding the first videoinput unit 123 and the second video input unit 124, shutter speed, or adiaphragm on the basis of the image signal or the taken image. The maincontroller 101 outputs the generated exposure information to the videoinput unit 120. Further, the main controller 101 causes the video signalprocessor 122 to generate the HDR synthetic image of the object on thebasis of a plurality of image signals whose exposure amounts aredifferent from each other.

Next, an imaging method according to the present embodiment will bedescribed. In the present embodiment, for example, various kinds ofprograms in the camera function program storage area 110 b or the likeillustrated in FIG. 3 are developed on the memory 104, and the maincontroller 101 executes each of the executing units such as the camerafunction executing unit 104 b, whereby an operation related to taking animage is executed.

FIG. 5 is a flowchart of the imaging method according to the firstembodiment of the present invention. As illustrated in FIG. 5, a videoinputting step S10, a motion information detecting step S20, an imagingstep S30, and an HDR synthetic image generating step S40 are executedwhen an image is taken.

When an instruction for taking an image is made from the operating unit140, the operation related to taking an image of the object is started.At the video inputting step S10, the video input unit 120 takes an imageof the object to generate an image signal of the object. The first videoinput unit 123 and the second video input unit 124 take an image of theobject, and generate electric signals as image signals in the firstimaging device 123 b and the second imaging device 124 b. The firstimaging device 123 b outputs the generated electric signal to the firstimage processor 122 a of the video signal processor 122, and the secondimaging device 124 b outputs the generated electric signal to the secondimage processor 122 b of the video signal processor 122. Here, the casewhere both the first video input unit 123 and the second video inputunit 124 take an image of the object has been explained. However, onlyany of the video input units may take an image of the object. Theprocessing flow then shifts to the motion information detecting stepS20.

FIG. 6 and FIG. 7 are a flowchart illustrating one example of processesat the motion information detecting step, the imaging step, the takenimage generating step and the like according to the first embodiment ofthe present invention. At the motion information detecting step S20, themain controller 101 detects the motion information of the object on thebasis of the image signal. Specifically, the first image processor 122 aof the video signal processor 122 outputs the electric signal outputtedfrom the first imaging device 123 b to the main controller 101. Thesecond image processor 122 b of the video signal processor 122 outputsthe electric signal outputted from the second imaging device 124 b tothe main controller 101. Note that the first image processor 122 a andthe second image processor 122 b may generate a taken image on the basisof the electric signal and output the generated taken image to the maincontroller 101.

The main controller 101 detects the motion information (for example,motion vector) of the object on the basis of these inputted electricsignals (image signals). The main controller 101 uses a well-known blockmatching method or the like for these image signals to detect the motionvector of the object. Note that the main controller 101 may detect themotion information of the object on the basis of the inputted takenimage. The processing flow then shifts to the imaging step S30.

At the imaging step S30, the video input unit 120 takes an image of theobject multiple times on the basis of the motion information detected atthe motion information detecting step S20 so as to differentiate anexposure amount thereof. At the imaging step S30, a process at Step S102is executed. At Step S102, motion of the object is determined on thebasis of the motion information. Specifically, the main controller 101compares the motion vector detected at the motion information detectingstep S20 with the first motion threshold value and the second motionthreshold value, thereby determining the motion of the object.Specifically, the main controller 101 reads out the first motionthreshold value and the second motion threshold value stored in thestorage 110 to the memory 104, and compares the motion vector with thefirst motion threshold value and the second motion threshold value.

[Still Image Imaging Mode]

In a case where the main controller 101 determines that the magnitude ofthe motion vector is less than the first motion threshold value at StepS102, a process at Step S113 is executed. Namely, the main controller101 determines that the object hardly moves, and switches into a stillimage imaging mode.

FIG. 8 is a view illustrating a timing chart of the imaging according tothe first embodiment of the present invention. FIG. 9 is a viewschematically illustrating an HDR synthesizing process according to thefirst embodiment of the present invention. At Step S113, for example, asillustrated in FIG. 8(A), the first video input unit 123 takes an imageof the object in succession so as to differentiate an exposure amountthereof. Specifically, the main controller 101 determines the exposureamount (for example, shutter speed, a diaphragm value and the like)whenever an image is taken on the basis of the electric signal (imagesignal) outputted from the first imaging device 123 b or the secondimaging device 124 b. For example, as illustrated in FIG. 8 (A), themain controller 101 sets the exosure amount on taking an image firsttime to an exposure amount L_(A0) at which a dark portion at a low lightquantity side does not become underexposure, and sets the exposureamount on taking an image second time to an exposure amount L_(A1) atwhich a bright portion at a high light quantity side does not becomeoverexposure, for example. The exposure amount L_(A1) is smaller thanthe exposure amount L_(A0) (L_(A0)>L_(A1)). The main controller 101outputs information regarding the determined exposure amount to thevideo input unit 120 as exposure information. In the video input unit120, on the basis of the inputted exposure information, the first videoinput unit 123 takes an image of the object with the exposure amountL_(A0) first time, and takes an image of the object with the exposureamount L_(A1) second time.

Whenever an image of the object is taken, the first imaging device 123 bgenerates an electric signal regarding the taken image as illustrated inFIG. 9, for example. Whenever an image of the object is taken, the firstimaging device 123 b generates an electric signal (A0) with the exposureamount L_(A0) and an electric signal (A1) with the exposure amountL_(A1) as illustrated in FIG. 9, for example, and outputs the generatedelectric signals (A0, A1) to the first image processor 122 a of thevideo signal processor 122. Here, the case where the image of the objectis taken twice so as to differentiate the exposure amount thereof hasbeen described. However, the image of the object may be taken threetimes or more, for example.

On the other hand, the second video input unit 124 takes a dynamic image(Bm) of the object, for example. Further, the second video input unit124 may take an image of the object so as to differentiate a depth offield thereof by changing diaphragm values, for example. The secondimaging device 124 b outputs an image signal regarding the dynamic imageand the like to the second image processor 122 b of the video signalprocessor 122. The processing flow then shifts to the HDR syntheticimage generating step S40.

At the HDR synthetic image generating step S40, a process at Step S114is executed. At Step S114, an HDR synthetic image with the firstresolution is generated on the basis of the plurality of image signals(A0, A1) whose exposure amounts are different from each other and thatis generated at Step S113. Specifically, the first image processor 122 aof the video signal processor 122 converts each of the electric signals(A0, A1) inputted from the first imaging device 123 b into digital imagedata with a gradation width of predetermined bits. The first imageprocessor 122 a then executes image processing for the respectivedigital image data to generate respective intermediate image datasuitable for HDR image synthesis. The first image processor 122 aoutputs the respective intermediate image data to the image synthesizer122 c. Further, the first image processor 122 a generates respectivetaken images before synthesis on the basis of the electric signals (A0,A1) Further, the first image processor 122 a generate thumbnail imagescorresponding to the respective taken images before synthesis.

The second image processor 122 b generates a dynamic image on the basisof the image signal inputted from the second video input unit 124, orgenerates a taken image so as to differentiate a depth of field thereof.Further, the second image processor 122 b generates thumbnail imagesrespectively corresponding to the generated dynamic image and thegenerated taken image.

The image synthesizer 122 c generates an HDR synthetic image (takenimage) (A0+A1) with the first resolution (high resolution) asillustrated in FIG. 9 by weighting and synthesizing the respectiveintermediate image data inputted from the first image processor 122 a,for example. The HDR synthetic image (A0+A1) generated in this mannerallows underexposure of a dark portion and/or overexposure of a brightportion to be solved, thereby becoming an image close to a scene thatthe user sees with the naked eye. Further, the image synthesizer 122 cgenerates a thumbnail image corresponding to the generated HDR syntheticimage.

Note that each of the first image processor 122 a and the second imageprocessor 122 b may convert an image signal with high resolution into animage signal with low resolution, and the image synthesizer 122 cgenerates an HDR synthetic image with low resolution. The processingflow then shifts to Step S115.

At Step S115, the HDR synthetic image generated at Step S114, the takenimages before syntheis, the dynamic image, the thumbnail image and thelike are stored in the storage 110. Specifically, the image synthesizer122 c outputs the generated HDR synthetic image (A0+A1) and thethumbnail image to the storage 110, and the storage 110 stores thereinthe inputted HDR image synthesis (A0+A1) and the thumbnail image.Further, each of the first image processor 122 a and the second imageprocessor 122 b outputs the taken image before synthesis, the dynamicimage, and the thumbnail image to the storage 110, and the storage 110stores therein the inputted taken images before synthesis, the dynamicimages, and the thumbnail images. When these processes are executed, theprocess at the HDR synthetic image Generating step S40 is completed, andthe processing flow shifts to Step S104. A process at Step S104 will bedescribed later.

[Micromotion Imaging Mode]

FIG. 10 and FIG. 11 are views schematically illustrating the HDRsynthesizing process according to the first embodiment of the presentinvention. In a case where the main controller 101 determines that themotion vector is equal to or more than the first motion threshold valueat Step S102 of the imaging step S30, a process at Step S123 or StepS133 is executed. Specifically, in a case where the main controller 101determines that the magnitude of the motion vector is equal to or morethan the first motion threshold value and is equal to or less than thesecond motion threshold value, the process at Step S123 is executed.Namely, the main controller 101 determines that the motion of the objectis greater compared with that in the still image imaging mode but themotion is smaller than a moving object imaging mode (will be describedlater), and switches into a micromotion imaging mode.

At Step S123, the main controller 101 causes the first video input unit123 and the second video input unit 124 to take an image of the objectat the same time so as to differentiate an exposure amount thereof, andthen causes the second video input unit 124 to take an image of theobject so as to differentiate the exposure amount thereof. Specifically,as illustrated in FIG. 8 (B), the main controller 101 sets the exposureamount on taking an image by the first video input unit 123 to theexposure amount L_(A0) at which a dark object at a low light quantityside does not become underexposure, for example. In other words, thefirst video input unit 123 takes an image of the object with suitableexposure. The suitable exposure herein means that an image of a darkobject at a low light quantity side is taken with a sufficient exposureamount. Further, the main controller 101 sets the exposure amount ontaking an image first time by the second video input unit 124 to anexposure amount L_(B0) between an exposure amount at which the low lightquantity side may become underexposure and an exposure amount at whichthe high light quantity side may become overexposure, for example, andsets the exposure amount on taking an image second time by the secondvideo input unit 124 to the exposure amount L_(A1) at which a brightportion does not become overexposure, for example. The exposure amountL_(A1) is smaller than exposure amount L_(B0) (L_(A0)>L_(B0)>L_(B1)).

Further, since resolution of the second imaging device 124 b is lowerthan that of the first imaging device 123 b, the second video input unit124 can acquire the same exposure amount even for a shorter imaging time(faster shutter speed) than that of the first video input unit 123.Therefore, since it is possible to shorten an imaging interval (timedifference) by the second video input unit 124 compared with the firstvideo input unit 123, it is possible to keep occurrence of noise due tomicromotion of the object down when the HDR synthesizing process isexecuted.

The main controller 101 outputs information regarding the determinedexposure amount to the video input unit 120 as exposure information. Inthe video input unit 120, on the basis of the inputted exposureinformation, the first video input unit 123 takes an image of the objectwith the exposure amount L_(A0), the second video input unit 124 takesan image of the object with the exposure amount L_(B0) first time, andtakes an image of the object with exposure amount L_(B1) second time.

The first imaging device 123 b generates the electric signal (A0) withthe exposure amount L_(A0) as illustrated in FIG. 10 (A), for example,and outputs the generated electric signal (A0) to the first imageprocessor 122 a of the video signal processor 122.

Whenever an image of the object is taken, the second imaging device 124b generates an electric signal (B0) of the exposure amount L_(B0) and anelectric signal (B1) of the exposure amount L_(B1) as illustrated inFIG. 10(A), for example, and outputs the generated electric signals (B0,B1) to the second image processor 122 b of the video signal processor122. The processing flow then shifts to the HDR synthetic imagegenerating step S40.

At the HDR synthetic image generating step S40, a process at Step S124is first executed. At Step S124, the resolution of the taken image ofthe electric signal generated at Step S123 is converted. Specifically,the main controller 101 causes the video signal processor 122 to convertthe resolution of the image signal generated by the first video inputunit 123 into the second resolution, and to convert the resolution ofthe image signal generated by the second video input unit 124 into thefirst resolution.

Specifically, the main controller 101 causes the first image processor122 a to convert resolution of the electric signal (A0) generated by thefirst imaging device 123 b from the first resolution to the secondresolution, and causes the second image processor 122 b to convertresolution of the electric signal (B0) generated by the second imagingdevice 124 b from the second resolution to the first resolution. Morespecifically, the first image processor 122 a executes resolutionconversion for the electric signal (A0) with the first resolution togenerate an electric signal (A0 d) with the second resolutionillustrated in FIG. 10(B). The second image processor 122 b executesresolution conversion for the electric signal (B0) with the secondresolution, and generates an electric signal (B0 u) with the firstresolution illustrated in FIG. 10(A) by executing a process such as aninterpolating process, for example. The processing flow then shifts toStep S125.

At Step S125, an HDR synthetic image with the first resolution and anHDR synthetic image with the second resolution are generated on thebasis of the plurality of image signals (A0, B0, B1, A0 d, B0 u)generated at Step S124. Specifically, the first image processor 122 aconverts each of the electric signal (A0) with the first resolution andthe electric signal (A0 d) with the second resolution into digital imagedata with a gradation width of predetermined bits. The first imageprocessor 122 a then executes image processing for the respectivedigital image data to generate intermediate image data suitable for HDRimage synthesis. The first image processor 122 a outputs the respectiveintermediate image data to the image synthesizer 122 c. Further, thefirst image processor 122 a generates respective taken images beforesynthesis on the basis of the electric signals (A0, A0 d). Further, thefirst image processor 122 a generates a thumbnail image corresponding toeach of the taken images before synthesis.

The second image processor 122 b converts each of the electric signals(B0, Bl) with the second resolution and the electric signal (B0 u) withthe first resolution into digital image data with a gradation width ofpredetermined bits. The second image processor 122 b then executes imageprocessing for the respective digital image data to generate respectiveintermediate image data suitable for HDR image synthesis. The secondimage processor 122 b outputs the respective intermediate image data tothe image synthesizer 122 c. Further, the second image processor 122 bgenerates respective taken images before synthesis on the basis of theelectric signals (B0, B1, B0 u). Further, the second image processor 122b generates thumbnail images respectively corresponding to the generatedtaken images before synthesis.

The image synthesizer 122 c generates an HDR synthetic image with thefirst resolution and an HDR synthetic image with the second resolutionby weighting and synthesizing the respective intermediate image dataoutputted from the first image processor 122 a and the respectiveintermediate image data outputted from the second image processor 122 b.

Specifically, the image synthesizer 122 c generates an HDR syntheticimage (A0+B0 u) with the first resolution illustrated in FIG. 10(A), forexample, on the basis of the intermediate image data of the electricsignal (A0) and the intermediate image data of the electric signal (B0u). Further, the image synthesizer 122 c generates an HDR syntheticimage (A0 d+B0) with the second resolution illustrated in FIG. 10(B) onthe basis of the intermediate image data of the electric signal (A0 d)and the intermediate image data of the electric signal (B0), forexample. Further, the image synthesizer 122 c generates an HDR syntheticimage (B0+B1) with the second resolution illustrated in FIG. 11(A) onthe basis of the intermediate image data of the electric signal (B0) andthe intermediate image data of the electric signal (B1), for example.Further, the image synthesizer 122 c generates an HDR synthetic image(A0 d+B0+B1) with the second resolution illustrated in FIG. 11(B) on thebasis of the intermediate image data of the electric signal (A0 d) andthe intermediate image data of the electric signals (B0, B1), forexample. Further, the image synthesizer 122 c generates a thumbnailimage corresponding to the generated HDR synthetic image.

When the user views or browses the HDR synthetic images later, the usermay select one, which is visually determined to be the best, from theseimages, and treats the one as an image of a final processing result bybeing subjected to the HDR synthesizing process. The processing flowthen shifts to Step S126.

At Step S126, the HDR synthetic images generated at Step S125, the takenimages before synthesis, the thumbnail images and the like are stored inthe storage 110. Specifically, the image synthesizer 122 c outputs thegenerated HDR synthetic images (A0+B0 u), (A0 d+B0), (B0+B1) and (A0d+B0+B1), and the thumbnail images to the storage 110, and the storage110 stores therein the inputted HDR image synthesis (A0+B0 u), (A0d+B0), (B0+B1) and (A0 d+B0+B1), and the thumbnail images. Further, eachof the first image processor 122 a and the second image processor 122 boutputs a taken image before synthesis and a thumbnail image to thestorage 110, and the storage 110 stores therein the inputted taken imagebefore synthesis and thumbnail image. When these processes are executed,the process at the HDR synthetic image generating step S40 is completed,and the processing flow then shifts to Step S104. A process at Step S104will be described later.

[Moving Object Imaging Mode]

At Step S102 of the imaging step S30, in a case where the maincontroller 101 determines that the motion vector exceeds the secondmotion threshold value, a process at Step S133 is executed. Namely, themain controller 101 determines that the motion of the object is greaterthan the micromotion imaging mode, and switches into a moving objectimaging mode.

At Step S133, a process similar to the process at Step S123 is executed.Namely, at Step S123, the main controller 101 causes the first videoinput unit 123 and the second video input unit 124 to take an image ofthe object at the same time so as to differentiate exposure amountsthereof. The first video input unit 123 takes an image of the objectwith the exposure amount L_(A0), for example, to generate the electricsignal (A0) for the taken image with the first resolution as illustratedin FIGS. 10(A) and 10(B). The second video input unit 124 takes an imageof the object with the exposure amount L_(B0), for example, to generatethe electric signal (B0) for the taken image with the second resolutionas illustrated in FIGS. 10(A) and 10(B). Note that in the moving objectimaging mode, the second video input unit 124 does not take an image ofthe object with time differences unlike the micromotion imaging mode.This is because the motion of the object is greater, and thus,deflection of the object in the generated HDR synthetic image becomesmarked when an image of the object is taken with time intervals.

The first imaging device 123 b outputs the electric signal (A0) of theexposure amount L_(A0) to the first image processor 122 a. The secondimaging device 124 b outputs the electric signal (B0) with the exposureamount L_(B0) to the second image processor 122 b. The processing flowthen shifts to the HDR synthetic image generating step S40.

At the HDR synthetic image generating step S40, a process at Step S134is first executed. At Step S134, the process similar to the process atStep S124 described above is executed. Namely, at Step S134, the firstimage processor 122 a executes resolution conversion for the electricsignal (A0) with the first resolution to generate the electric signal(A0 d) with the second resolution illustrated in FIG. 10(B), forexample. The second image processor 122 b executes resolution conversionfor the electric signal (B0) with the second resolution to generate theelectric signal (B0 u) with the first resolution illustrated in FIG.10(A). The processing flow then shifts to Step S135.

At Step S135, a process substantially similar to the process at StepS125 described above is executed. Namely, at Step S135, an HDR syntheticimage with the first resolution and an HDR synthetic image with thesecond resolution are generated on the basis of the plurality of imagesignals (A0, B0, A0 d, B0 u) generated at Step S134. Specifically, theimage synthesizer 122 c generates the HDR synthetic image (A0+B0 u) withthe first resolution illustrated in FIG. 10(B) and the HDR syntheticimage (A0 d+B0) with the second resolution illustrated in FIG. 10(A).Further, each of the first image processor 122 a and the second imageprocessor 122 b generates a taken image before synthesis and a thumbnailimage corresponding to the taken image before synthesis.

When the user views or browses the HDR synthetic images later, the usermay select one, which is visually determined to be the best, from theseimages, and treats the one as an image of a final processing result bybeing subjected to the HDR synthesizing process. The processing flowthen shifts to Step S126.

At Step S126, the HDR synthetic images generated at Step S135, the takenimages before synthesis, the thumbnail images and the like are stored inthe storage 110. Specifically, the image synthesizer 122 c outputs thegenerated HDR synthetic images (A0+B0 u) and (A0 d+B0), and thethumbnail images to the storage 110, and the storage 110 stores thereinthe inputted HDR image synthesis (A0+B0 u) and (A0 d+B0), and thethumbnail images. Further, each of the first image processor 122 a andthe second image processor 122 b outputs a taken image before synthesisand a thumbnail image to the storage 110, and the storage 110 storestherein the inputted taken image before synthesis and thumbnail image.When these processes are executed, the process at the HDR syntheticimage generating step S40 is completed, and the processing flow thenshifts to Step S104.

At Step S104, an image to be displayed on the display 121 and the likeare selected. Specifically, the image to be displayed on the display 121and the like are selected among the images generated at each of StepsS114, S125, S135 for the respective imaging modes and stored in thestorage 110 (including the HDR synthetic image, the taken image beforesynthesis, and the dynamic image).

For example, the storage 110 outputs information regarding the storedimages (for example, the thumbnail images) to the image output unit 122d. The image output unit 122 d outputs the inputted thumbnail images tothe display 121. The display 121 displays the inputted thumbnail imagesthereon. The user selects the image or the like displayed on the display121 from the displayed thumbnail images. The processing flow then shiftsto Step S105.

At Step S105, the image or the like corresponding to the selectedthumbnail image is displayed. Specifically, the image output unit 122 doutputs the image or the like corresponding to the thumbnail imageselected by the user from the storage 110, for example. The image outputunit 122 d outputs the image read out therefrom to the display 121. Thedisplay 121 displays the inputted image or the like.

Note that the user may arbitrarily select the image or the like to bedisplayed on the display 121, the preference order to be displayed maybe registered in advance in the imaging apparatus 100, and the imagesand the like may be displayed in turn in accordance with this order.

Further, the image and the like to be displayed on the display 121 maybe expanded or reduced appropriately in accordance with resolution ofthe display 121. In this case, a plurality of images or the like may bedisplayed with the same size, or the plurality of images or the like maybe displayed with respective different sizes in accordance withresolution of the selected image or the like.

Note that the imaging apparatus 100 according to the present embodimentis not limited to the configuration illustrated in FIG. 2. For example,the imaging apparatus 100 may not include the communication processor150 or the sensor unit 160, or may include various kinds of functionssuch as a digital television broadcasting receiving function or anelectronic money settlement function.

According to the present embodiment, on the basis of the motion vectorof the object, the main controller 101 causes the video input unit 120to take an image of the object multiple times so as to differentiate anexposure amount thereof, and causes the video signal Processor 122 togenerate an HDR synthetic image of the object on the basis of theplurality of image signals whose exposure amounts are different fromeach other.

According to this configuration, it is possible to select an appropriateimaging mode based on motion of the object. Therefore, it is possible togenerate a high-quality HDR synthetic image compatible with imagingenvironment such as motion of the object or camera shake.

Further, according to the present embodiment, in a case where the maincontroller 101 determines that the magnitude of the motion vector isless than the first motion threshold value, the main controller 101switches into the still image imaging mode. The main controller 101causes the first video input unit 123 including the first imaging device123 b first resolution that is high resolution to take an image of theobject in succession so as to differentiate an exposure amount thereof,and causes the video signal processor 122 to generate an HDR syntheticimage with the first resolution on the basis of the plurality of imagesignals (A0, A1) whose exposure amounts are different from each other.

According to this configuration, the object hardly moves in the stillimage imaging mode. Therefore, even though an image of the object istaken with time intervals, it is possible to keep occurrence of noisedue to motion of the object down in the HDR synthetic image. This makesit possible to take an image of the object in succession by using onlyan imaging device with high resolution. Therefore, it is possible togenerate a high-quality HDR synthetic image.

Further, according to this configuration, the electric signal (A0)generated by taking an image of a dark object at a low light quantityside with a sufficient exposure amount and the electric signal (A1)generated by taking an image of a bright object at a high light quantityside while suppressing the exposure amount are synthesized. Therefore, ahigh-quality HDR synthetic image whose dynamic range is expanded isgenerated.

Further, according to the present embodiment, in a case where the maincontroller 101 determines that the motion vector is equal to or morethan the first motion threshold value, the main controller 101 switchesinto the micromotion imaging mode or the moving object imaging mode. Themain controller 101 causes the first video input unit 123 and the secondvideo input unit 124 to take an image of the object at the same time soas to differentiate exposure amounts thereof; and causes the videosignal processor 122 to convert the image signal with the firstresolution into the image signal with the second resolution and toconvert the image signal with the second resolution into the imagesignal with the first resolution to generate the HDR synthetic imagewith the first resolution and the HDR synthetic image with the secondresolution.

According to this configuration, even in a case where the object moves,the HDR synthetic image is Generated on the basis of the plurality ofimage signals obtained by taking an image at the same time. Therefore,it is possible to suppress an influence of noise caused by motion of theobject or camera shake, and a high-quality HDR synthetic image in whicha dynamic range thereof is expanded is generated.

Further, according to this configuration, since the plurality of HDRimages whose resolution is different from each other is generated, it ispossible to provide the HDR images suitable for use application of theuser.

Further, according to the present embodiment, in a case where the maincontroller 101 determines that the motion vector is equal to or morethan the first motion threshold value and is equal to or less than thesecond motion threshold value, the main controller 101 switches into themicromotion imaging mode. The main controller 101 causes the first videoinput unit 123 and the second video input unit 124 to take an image ofthe object at the same time so as to differentiate exposure amountsthereof, and then causes the second video input unit 124 to take animage of the object so as to differentiate an exposure amount thereof.

According to this configuration, the HDR synthesizing process isexecuted on the basis of at least three kinds of image signals whoseexposure amounts are different from each other. Therefore, it ispossible to generate the HDR synthetic image with higher quality.

Further, according to the present embodiment, the resolution of thesecond imaging device 124 b is lower than the resolution of the firstimaging device 123 b.

According to this configuration, the second video input unit 124 canacquire the same exposure amount even in the imaging time shorter thanthat of the first video input unit 123. This makes it possible tosuppress occurrence of noise due to motion of the object even though thesecond video input unit 124 takes an image of the object with timeintervals. Therefore, it is possible to suppress occurrence of noise dueto motion of the object in the HDR synthetic image.

Further, according to this configuration, it is possible to lower thecost of the second imaging device 124. This makes it possible to lower amanufacturing cost of the imaging apparatus 100.

Further, according to the present embodiment, the display 121 displaysthe thumbnail image such as the HDR synthetic image. When the thumbnailimage is selected, the display 121 displays the HDR synthetic imagecorresponding to the selected thumbnail image.

According to this configuration, the user is allowed to readily identifythe HDR synthetic images and the like stored in the storage 110.Therefore, it is possible to reduce a load when the user selects the HDRimage or the like to be displayed on the display 121.

Further, according to the present embodiment, the imaging methodincludes: the video inputting step S10 of causing the video input unit120 to take an object of an object to generate an image signal of theobject; the motion information detecting step S20 of detecting motioninformation of the object on the basis of the image signal by the maincontroller 101; the imaging step S30 of causing the video input unit 120to take an image of the object multiple times on the basis of the motioninformation so as to differentiate an exposure amount thereof by themain controller 101; and the HDR synthetic image generating step S40 ofcausing the video signal processor 122 to generate an HDR syntheticimage of the object on the basis of a plurality of image signals whoseexposure amounts are different from each other.

According to this configuration, it is possible to select an appropriateimaging mode based on motion of the object and take an image of theobject. Therefore, it is possible to provide an imaging method capableof generating a high-quality HDR synthetic image compatible with imagingenvironment.

Further, according to the present embodiment, the main controller 101that is a computer is caused to execute: the video inputting step S10 ofcausing the video input unit 120 to take an object of an object togenerate an image signal of the object; the motion information detectingstep S20 of detecting motion information of the object on the basis ofthe image signal; the imaging step S30 of causing the video input unit120 to take an image of the object multiple times on the basis of themotion information so as to differentiate an exposure amount thereof;and the HDR synthetic image generating step S40 of causing the videosignal processor 122 to generate an HDR synthetic image of the object onthe basis of a plurality of image signals whose exposure amounts aredifferent from each other.

According to this configuration, it is possible to select an appropriateimaging mode based on motion of the object and take an image of theobject. Therefore, it is possible to provide an imaging program capableof Generating a high-quality HDR synthetic image compatible with imagingenvironment.

Second Embodiment

Next, a second embodiment will be described. In the present embodiment,a case where imaging modes are switched on the basis of motion and lightquantity of an object will be described. Hereinafter, detailedexplanation of overlapped portions with those according to the firstembodiment described above may be omitted appropriately.

FIG. 12 and FIG. 13 are block diagrams illustrating one example of aconfiguration of an imaging apparatus according to the second embodimentof the present invention. As illustrated in FIG. 12, an imagingapparatus 200 includes a video input unit 220 and the like.

As illustrated in FIG. 12 and FIG. 13, the video input unit 220 includesa second video input unit 224, a resolution converter 228, and the like.The second video input unit 224 includes a second imaging device 224 b.The second imaging device 224 b is an imaging device with firstresolution (high resolution) as well as the first imaging device 123 b.The first imaging device 123 b and the second imaging device 224 b areconfigured so that resolution is converted appropriately.

The light quantity sensor 165 measures light quantity in the vicinity ofthe periphery of the imaging apparatus 200 or the object. The lightquantity sensor 165 then outputs the measured light quantity informationto the exposure controller 126 and the main controller 101.

For example, the exposure controller 126 sets an exposure amount whenthe first video input unit 123 and the second video input unit 224 takean image on the basis of the light quantity information outputted fromthe light quantity sensor 165 and an instruction from the maincontroller 101.

The main controller 101 switches imaging modes on the basis of the lightquantity measured by the light quantity sensor 165 and a detectingresult of motion vector of the object. Further, the main controller 101causes the resolution converter 228 to group a plurality of pixels,thereby converting resolution of each of the first imaging device 123 band the second imaging device 224 b from the first resolution (highresolution) to second resolution (low resolution). Further, the maincontroller 101 causes the resolution converter 228 to release thegrouping of the plurality of images, thereby converting the resolutionof each of the first imaging device 123 b and the second imaging device224 b from the second resolution (low resolution) to the firstresolution (high resolution).

For example, the main controller 101 outputs resolution conversioninforma ion to the resolution converter 228. The resolution conversioninformation is used to convert the resolution of each of the firstimaging device 123 b and the second imaging device 224 b in accordancewith the imaging mode. The resolution converter 228 converts theresolution of each of the first imaging device 123 b and the secondimaging device 224 b on the basis of the resolution conversioninformation. In the imaging device whose resolution is converted intolow resolution, a plurality of grouped pixels is set to one pixel.Herewith, an exposure amount in one pixel after grouping is increasedcompared with one pixel before Grouping. Therefore, it is possible toshorten an exposure time (shutter speed) to acquire the same exposureamount.

The light quantity sensor 165 measures light quantity of the object froma region of high luminosity to a region of low luminosity. FIG. 12 andFIG. 13 illustrate a case where the light quantity sensor 165 isprovided independently. However, the light quantity sensor 165 is notlimited to such a configuration. For example, the first imaging device123 b or the second imaging device 224 b may also be provided with afunction of the light quantity sensor 165.

Next, an imaging method according to the present embodiment will bedescribed. FIG. 14 and FIG. 15 are flowcharts related to the imagingmethod according to the second embodiment of the present invention. Inthe present embodiment, the video inputting step S10, the motioninformation detecting step S20, the imaging step S30, and the HDRsynthetic image generating step S40, which are illustrated in FIG. 5,are executed. The video inputting step S10 and the motion informationdetecting step S20 are similar to those according to the firstembodiment described above.

At the imaging step S30, a process at Step S202 is first executed. AtStep S202, motion of the object is determined on the basis of the motioninformation. Specifically, the main controller 101 compares the motionvector detected at the motion information detecting step S20 with athird motion threshold value, thereby determining the motion of theobject. Specifically, the main controller 101 reads out the third motionthreshold value stored in the storage 110 to the memory 104, andcompares the motion vector with the third motion threshold value. Thethird motion threshold value is set appropriately y experiments and thelike, for example.

[Still Image imaging Mode]

In a case where the main controller 101 determines that the magnitude ofthe motion vector is less than the third motion threshold value at StepS202, a process at Step S214 is executed. Namely, the main controller101 determines that the object hardly moves, and switches into a stillimage imaging mode.

At Step S214, a process similar to the process at Step S113 according tothe first embodiment is executed. Namely, the first video input unit 123takes an image of the object in succession so as to differentiate anexposure amount thereof. The first video input unit 123 takes an imageof the object with an exposure amount L_(A0) on taking an image firsttime, for example, and takes an image of the object with an exposureamount L_(A1), which is smaller than the exposure amount L_(A0)(L_(A0)>L_(A1)), on taking an image second time. For example, the firstimaging device 123 b generates electric signals (A0, A1) illustrated inFIG. 9 with respect to a taken image with high resolution, and outputsthe generated electric signals (A0, A1) to the first image processor 122a. Here, a case where two kinds of image signals whose exposure amountsare different from each other are subjected to a synthesizing processwill be described. However, for example, three kinds or more of imagesignals may be subjected to the synthesizing process.

On the other hand, the second video input unit 224 may take a dynamicimage (Bm) of the object, for example, or take an image of the object soas to differentiate a depth of field thereof. Further, since the secondimaging device 224 b of the second video input unit 224 is also highresolution, for example, the second video input unit 224 may take animage of the object at the same time with the first video input unit 123so as to differentiate an exposure amount thereof (for example, L_(A1)or the like). The second imaging device 224 b generates the electricsignal (A0) with the exposure amount L_(A0) and an electric signal (A1)with the exposure amount L_(A1), and outputs the generated electricsignals (A0, A1) to the first image processor 122 a. The second imagingdevice 224 b outputs the image signals regarding the dynamic image andthe like to the second image processor 122 b of the video signalprocessor 122. The processing flow then shifts to the HDR syntheticimage generating step S40.

At the HDR synthetic image generating step S40, a process at Step S215is executed. At Step S215, a process similar to the process at Step S114according to the first embodiment is executed. Namely, the imagesynthesizer 122 c generates an HDR synthetic image (A0+A1) with thefirst resolution on the basis of the plurality of image signals (A0, A1)whose exposure amounts are different from each other, which aregenerated at Step S214. Further, the image synthesizer 122 c maygenerate an HDR synthetic image on the basis of the electric signalsgenerated by the first imaging device 123 b and the second imagingdevice 224 b. Further, the image synthesizer 122 c generates a thumbnailimage corresponding to the HDR synthetic image. The first imageprocessor 122 a and the second image processor 122 b generate takenimages before synthesis, dynamic images, and thumbnail imagescorresponding to these images.

Note that the first image processor 122 a and the second image processor122 b may convert an image signal with high resolution into an imagesignal with low resolution, and the image synthesizer 122 c may generatean HDR synthetic image with low resolution. The processing flow thenshifts to Step S216.

At Step S216, a process similar to the process at Step S115 according tothe first embodiment is executed. Namely, the HDR synthetic imagegenerated at Step S215, the taken images before synthesis, the dynamicimages, the thumbnail images and the like are stored in the storage 110.

[Moving Object Imaging Mode]

FIG. 16 is a view schematically illustrating an HDR synthesizing processaccording to the second embodiment of the present invention. In a casewhere the main controller 101 determines at Step S202 of the imagingstep S30 that the motion vector is equal to or more than the thirdmotion threshold value, the main controller 101 determines that motionof the object is large, and switches into a moving object imaging mode.When to switch into the moving object imaging mode, the processing flowshifts to Step S203.

At Step S203, the imaging modes are switched on the basis of the lightquantity measured by the light quantity sensor 165. Specifically, thelight quantity sensor 165 measures light quantity of the periphery ofthe imaging apparatus 200, and outputs information regarding themeasured light quantity to the main controller 101 as light quantityinformation. The main controller 101 compares the measured lightquantity with a first light quantity threshold value and a second lightquantity threshold value on the basis of the inputted light quantityinformation. The second light quantity threshold value is larger thanthe first light quantity threshold value. Specifically, the maincontroller 101 reads out the first light quantity threshold value andthe second light quantity threshold value stored in the storage 110 tothe memory 104, and compares the light quantity with the first lightquantity threshold value and the second light quantity threshold value.The first light quantity threshold value and the second light quantitythreshold value are set appropriately by experiments and the like, forexample.

<High Light Quantity Moving Object Imaging Mode>

In a case where the main controller 101 determines that the lightquantity exceeds the second light quantity threshold value, theprocessing flow shifts to Step S224. Namely, the main controller 101determines that the light quantity is high, and switches into a highlight quantity moving object imaging mode.

At Step S224, the main controller 101 causes the first video input unit123 and the second video input unit 224 to take an image of the objectat the same time so as to differentiate exposure amounts thereof. Atthis time, the main controller 101 sets resolution of each of the firstimaging device 123 b and the second imaging device 224 b to the firstresolution that is high resolution.

For example, in a case where the resolution of the first imaging device123 b or the second imaging device 224 b is set to low resolution, themain controller 101 outputs the resolution conversion information, bywhich resolution of the corresponding imaging device is to be converted,to the resolution converter 228. The resolution converter 228 convertsthe resolution of the corresponding imaging device from low resolutionto high resolution on the basis of the inputted resolution conversioninformation.

When the resolution of the imaging device is set in this manner, thefirst video input unit 123 takes an image of the object with theexposure amount L_(A0), for example, to generate the electric signal(A0) as illustrated in FIG. 16 with respect to the taken image with highresolution. The second video input unit 124 takes an image of the objectwith an exposure amount L_(B0) that is smaller than the exposure amountL_(A0) (L_(A0)>L_(B0)) for example, to generate an electric signal (B0)as illustrated in FIG. 16 with respect to the taken image with highresolution.

The first imaging device 123 b outputs the electric signal (A0) with theexposure amount L_(A0) to the first image processor 122 a. The secondimaging device 124 b outputs the electric signal (B0) with the exposureamount L_(B0) to the second image processor 122 b. The processing flowthen, shifts to the HDR synthetic image generating step S40.

At the HDR synthetic image generating step S40, a process at Step S225is executed. At Step S225, the image synthesizer 122 c generates an HDRsynthetic image (A0+B0) with high resolution illustrated in FIG. 16 onthe basis of the plurality of image signals (A0, B0) generated at StepS224, for example. Further, the image synthesizer 122 c generates athumbnail image corresponding to the HDR synthetic image. The firstimage processor 122 a and the second image processor 122 b generates thetaken images before synthesis, the dynamic images, and the thumbnailimages corresponding to these images.

Note that each of the first image processor 122 a and the second imageprocessor 122 b may converts an image signal with high resolution intoan image signal with low resolution, and the image synthesizer 122 c maygenerate an HDR synthetic image with low resolution. The Processing flowthen shifts to Step S237.

At Step S237, a process similar to the process at Step S126 according tothe first embodiment is executed. Namely, the HDR synthetic imagegenerated at Step S225, the taken image before synthesis, the dynamicimage, the thumbnail image and the like are stored in the storage 110.The processing flow then shifts to Step S204. The process at Step S204will be described later.

<Intermediate Light Quantity Moving Object imaging Mode>

FIG. 17 is a view schematically illustrating an HDR synthesizing processaccording to the second embodiment of the present invention. In a casewhere the main controller 101 determines that the light quantity isequal to or more than the first light quantity threshold value and isequal to or less than the second light quantity threshold value, theprocessing flow shifts to Step S234. Namely, the main controller 101determines that the light quantity is smaller than that in the highlight quantity moving object imaging mode, and switches into anintermediate light quantity moving object imaging mode.

At Step S234, the main controller 101 causes the first video input unit123 and the second video input unit 224 to take an image of the objectat the same time so as to differentiate exposure amounts thereof. Atthis time, the main controller 101 causes the resolution converter 228to convert resolution of the first imaging device 123 b into the firstresolution, and to convert resolution of the second imaging device 224 binto the second resolution.

For example, in a case where the resolution of the first imaging device123 b is set to low resolution, the main controller 101 outputs theresolution conversion information, by which the resolution of the firstimaging device 123 b is to be converted, to the resolution converter228. The resolution converter 228 converts the resolution of the firstimaging device 123 b from low resolution to high resolution on the basisof the inputted resolution conversion information. Further, in a casewhere the resolution of the second imaging device 224 b is set to highresolution, the main controller 101 outputs the resolution conversioninformation, by which the resolution of the second imaging device 224 bis to be converted, to the resolution converter 228. The resolutionconverter 228 converts the resolution of the second imaging device 224 bfrom high resolution to low resolution on the basis of the inputtedresolution conversion information.

When the resolution of the imaging device is set in this manner, thefirst video input unit 123 takes an image of the object with theexposure amount L_(A0), for example, to generate the electric signal(A0) as illustrated in FIGS. 17(A) and 17(B) with respect to the takenimage with high resolution. The second video input unit 124 takes animage of the object with an exposure amount L_(b0) that is smaller thanthe exposure amount L_(A0) (L_(A0)>L_(b0)), for example, to generate anelectric signal (b0) as illustrated in FIGS. 17(A) and 17(B) withrespect to the taken image with low resolution. Since resolution of thesecond video input unit 224 is smaller than that of the first videoinput unit 123, it is possible to appropriately set the exposure amountfor expanding a dynamic range in the HDR synthetic image.

The first imaging device 123 b outputs the electric signal (A0) with theexposure amount L_(A0) to the first image processor 122 a. The secondimaging device 124 b outputs the electric signal (b0) of the exposureamount L_(b0) to the second image processor 122 b. The processing flowthen shifts to the HDR synthetic image generating step S40.

At the HDR synthetic image generating step S40, a process at Step S235is first executed. At Step S235, for example, the process similar to theprocess at Step S124 according to the first embodiment is executed.Namely, resolution of each of the electric signals generated at StepS235 and Step S234 is converted. Specifically, the first image processor122 a executes resolution conversion for the electric signal (A0) withhigh resolution, for example, to generate an electric signal (A0 d) withlow resolution illustrated in FIG. 17(B). The second image processor 122b executes resolution conversion for the electric signal (b0) with lowresolution, for example, to generate an electric signal (B0 u) with highresolution illustrated in FIG. 17(A). The processing flow then shifts toStep S236.

At Step S236, the image synthesizer 122 c generates an HDR syntheticimage (A0+B0 u) with high resolution illustrated in FIG. 17(A), forexample, and an HDR synthetic image (A0 d+b0) with low resolutionillustrated in FIG. 17(B), for example, on the basis of the plurality ofelectric signals (A0, b0, A0 d, B0 u) generated at Step S235. Further,the image synthesizer 122 c generates a thumbnail image corresponding tothe HDR synthetic image. The first image processor 122 a and the secondimage processor 122 b generates the taken images before synthesis, thedynamic images, and the thumbnail images corresponding to these images.The processing flow then shifts to Step S237.

At Step S237, the HDR synthetic images generated at Step S236, the takenimages before synthesis, the dynamic image, the thumbnail images and thelike are stored in the storage 110. The processing flow then shifts toStep S204. The process at Step S204 will be described later.

<Low Light Quantity Moving Object Imaging Mode>

FIG. 18 is a view schematically illustrating an HDR synthesizing processaccording to the second embodiment of the present invention. In a casewhere the main controller 101 determines that the light quantity is lessthan the first light quantity threshold value, the processing flowshifts to Step S244. Namely, the main controller 101 determines that thelight quantity is smaller than that in the intermediate light quantitymoving object imaging mode, and switches into a low light quantitymoving object imaging mode.

At Step S224, the main controller 101 causes the first video input unit123 and the second video input unit 224 to take an image of the objectat the same time so as to differentiate exposure amounts thereof. Atthis time, the main controller 101 causes the resolution converter 228to convert resolution of each of the first imaging device 123 b and thesecond imaging device 224 b into the second resolution that is lowresolution.

For example, in a case where the resolution of the first imaging device123 b or the second imaging device 224 b is set to high resolution, themain controller 101 outputs the resolution conversion information, bywhich the resolution of the corresponding imaging device is to beconverted, to the resolution converter 228. The resolution converter 228converts the resolution of the corresponding imaging device from highresolution to low resolution on the basis of the inputted resolutionconversion information.

When the resolution of the imaging device is set in this manner, thefirst video input unit 123 takes an image of the object with theexposure amount L_(a0), for example, to generate the electric signal(a0) as illustrated in FIG. 18 with respect to the taken image with lowresolution. The second video input unit 124 takes an image of the objectwith an exposure amount L_(b0) that is smaller than the exposure amountL_(a0), for example, to generate an electric signal (b0) as illustratedin FIG. 18 with respect to the taken image with low resolution. Sinceresolution of each of the first video input unit 123 and the secondvideo input unit 224 is set to low resolution, it is possible toappropriately set the exposure amount for expanding the dynamic range inthe HDR synthetic image.

The first imaging device 123 b outputs the electric signal (a0) with theexposure amount L_(a0) to the first image processor 122 a. The secondimaging device 124 b outputs the electric signal (b0) of the exposureamount L_(b0) to the second image processor 122 b. The processing flowthen shifts to the HDR synthetic image generating step S40.

At the HDR synthetic image generating step S40, a process at Step S225is executed. At Step S225, the image synthesizer 122 c generates an HDRsynthetic image (a0+b0) with low resolution illustrated in FIG. 18 onthe basis of the plurality of image signals (a0, b0) generated at StepS224, for example. Further, the image synthesizer 122 c generates athumbnail image corresponding to the HDR synthetic image. The firstimage processor 122 a and the second image processor 122 b generates thetaken images before synthesis, the dynamic images, and the thumbnailimages corresponding to these images. The processing flow then shifts toStep S237.

At Step S237, the HDR synthetic image generated at Step S225, the takenimage before synthesis, the dynamic image, the thumbnail image and thelike are stored in the storage 110.

Note that each of the first image processor 122 a and the second imageprocessor 122 b may convert an image signal with low resolution into animage signal with high resolution, and the image synthesizer 122 c maygenerate an HDR synthetic image with high resolution. The processingflow then shifts to Step S204.

At Step S204, a process similar to the process at Step S104 according tothe first embodiment is executed, whereby an image or the like to bedisplayed on the display 121 is selected. The processing flow thenshifts Step S205.

At Step S205, a process similar to the process at Step S105 according tothe first embodiment is executed, whereby the image selected by a userat Step S204 is displayed on the display 121.

According to the present embodiment, the imaging apparatus 100 includesthe light quantity sensor 165 configured to measure light quantity, andthe main controller 101 causes the video input unit 220 to take an imageof the object multiple times so as to differentiate an exposure amountthereof on the basis of the motion vector and the light quantity.

According to this configuration, it is possible to select a moreappropriate imaging mode based on motion of the object and lightquantity of the periphery. Therefore, it is possible to generate ahigh-quality HDR synthetic image compatible with imaging environment.

Further, according to the present embodiment, the main controller 101compares the motion vector with the third motion threshold value. In acase where it is determined that the motion information is less than thethird motion threshold value, the main controller 101 switches into thestill image imaging mode. The main controller 101 causes the first videoinput unit 123 to take an image of the object in succession so as todifferentiate an exposure amount thereof, and causes the video signalprocessor 122 to generate the HDR synthetic image with the firstresolution.

According to this configuration, since the object hardly moves, it ispossible to suppress occurrence of noise due to motion of the objecteven in a case of taking an image of the object with time intervals.This makes it possible to generate a high-quality HDR synthetic imagecompatible with imaging environment by using only the first video inputunit 123. Further, this makes it possible to use the second video inputunit 224 for motion or animation imaging, and the imaging apparatus 200with excellent handleability is thus provided.

Further, according to the present embodiment, in a case where the maincontroller 101 determines that the motion vector is equal to or morethan the third motion threshold value and the light quantity exceeds thesecond light quantity threshold value, the main controller 101 switchesinto the high light quantity moving object imaging mode, causes thefirst video input unit 123 and the second video input unit 224 to takean image of the object at the same time so as to differentiate exposureamounts thereof, and causes the video signal processor 122 to generatethe HDR synthetic image with high resolution.

According to this configuration, it is possible to suppress occurrenceof noise due to motion of the object. Therefore, the high-quality HDRsynthetic image (A0+B0) with high resolution is generated even thoughmotion of the object is large.

Further, according to the present embodiment, in a case where the maincontroller 101 determines that the motion vector is equal to or morethan the third motion threshold value, the light quantity is equal to ormore than the first light quantity threshold value and is equal to orless than the second light quantity threshold value, then the maincontroller 101 switches into the intermediate light quantity movingobject imaging mode. The main controller 101 causes the resolutionconverter 228 to convert resolution of the second imaging device 224 binto the second resolution, and causes the first video input unit 123and the second video input unit 224 to take an image of the object atthe same time so as to differentiate exposure amounts thereof. The maincontroller 101 then converts resolution of the image signal generated bythe first video input unit 123 into the second resolution, convertsresolution of the image signal generated by the second video input unit224 into the first resolution, and causes the video signal processor 122to generate an HDR synthetic image with the first resolution and an HDRsynthetic image with the second resolution.

According to this configuration, by converting the resolution of thesecond imaging device 224 b into low resolution, an area of each of thepixels after resolution conversion is increased. Therefore, it ispossible to improve sensitivity of each of the pixels. This makes itpossible to shorten an imaging time (exposure time) by the second videoinput unit 224. Therefore, even in a situation that light quantity issmall, it is possible to suppress occurrence of noise due to motion ofthe object, and it is possible to generate the high-quality HDRsynthetic images (A0+B0 u, A0 d+B0) compatible with imaging environment.

Further, according to this configuration, since a plurality of HDRsynthetic images each of which has different resolution is generated,the imaging apparatus 200 with excellent handleability is provided.

Further, according to the present embodiment, in a case where the maincontroller 101 determines that the motion vector is equal to or morethan the third motion threshold value and the light quantity is lessthan the first light quantity threshold value, the main controller 101switches into the low light quantity moving object imaging mode. Themain controller 101 causes the resolution converter 228 to convert theresolution of each of the first imaging device 123 b and the secondimaging device 224 b into the second resolution, causes the first videoinput unit 123 and the second video input unit 224 to take an image ofthe object at the same time so as to differentiate exposure amountsthereof, and causes the video signal processor 122 to aenerate an HDRsynthetic image with the second resolution.

According to this configuration, by converting the resolution of each ofthe first imaging device 123 b and the second imaging device 224 b intolow resolution, an area of each of the pixels after resolutionconversion is increased. Therefore, it is possible to improvesensitivity of each of the pixels. This makes it possible to shorten theimaging time of both the first video input unit 123 and the second videoinput unit 224. Therefore, even in a situation that light quantity isfurther small, it is possible to suppress occurrence of noise due tomotion of the object, and it is possible to generate the high-qualityHDR synthetic images (a0+B0) compatible with imaging environment.

Further, according to the present embodiment, the main controller 101causes the resolution converter 228 to group a plurality of pixels,thereby converting the resolution of each of the first imaging device123 b and the second imaging device 224 b from high resolution to lowresolution. Further, the main controller 101 causes the resolutionconverter 228 to release grouping of the plurality of images, therebyconverting the resolution of each of the first imaging device 123 b andthe second imaging device 224 b from low resolution to high resolution.

According to this configuration, since there is no need to prepareplural types of imaging devices for plural kinds of resolution, it ispossible to miniaturize the imaging apparatus. This makes is possible tosuppress manufacturing cost of the imaging apparatus from beingincreased.

Another Embodiment

In the first and second embodiments that have been explained above, amain controller 101 detects motion information of an object (forexample, motion vector) on the basis of an image signal generated by afirst imaging device 123 b or a second imaging device 124 b (or 224 b).However, in addition to this, for example, the main controller 101 maydetect the motion information of the object on the basis of a focaldistance of a video input unit 120. Specifically, in a case where thefocal distance of the video input unit 120 is short, an influence ofcamera shake becomes small. In a case where the focal distance is long,the influence of the camera shake becomes larger. Namely, the maincontroller 101 detects motion information indicating that the motion ofthe object is small when the focal distance is short. The maincontroller 101 detects motion information indicating that the motion ofthe object is large when the focal distance is long. Thus, the maincontroller 101 compares the focal distance of the video input unit 120with a first focal distance threshold value and a second focal distancethreshold value. The second focal distance threshold value is largerthan the first focal distance threshold value. Then, in a case where themain controller 101 determines that the focal distance of the videoinput unit 120 is less than the first focal distance threshold value,the main controller 101 switches into a still image imaging mode.Further, in a case where the main controller 101 determines that thefocal distance of the video input unit 120 is equal to or more than thefirst focal distance threshold value and is equal to or less than thesecond focal distance, the main controller 101 switches into amicromotion imaging mode. Further, in a case where the main controller101 determines that the focal distance of the video input unit 120exceeds the second focal distance threshold value, the main controller101 switches into a moving object imaging mode. Even though the imagingmodes are switched on the basis of the focal distance of the video inputunit 120 in this manner, the effects described above can be obtained.

Moreover, in the first and second embodiments, the case where an imageof the object is taken by two cameras, that is, the first video inputunit 123 and the second video input unit 124 (or 224) to generate theHDR synthetic image has been described. However, for example, an imageof the object may be taken by using three or more cameras to Generate anHDR synthetic image.

As described above, the embodiments of the present invention have beenexplained, but needless to say, the configuration to realize thetechnique of the present invention is not limited these embodiments.Further, the numerical values and the like described in thespecification or illustrated in the drawings are just one example. Eventhough different numerical values are utilized, the effects of thepresent invention are not lost.

A part or all of the functions and the like of the present inventiondescribed above may be realized with hardware by designing an integratedcircuit, for example. A computer such as a microprocessor unit mayinterpret programs that realize the respective functions and execute theprograms, thereby realizing a part or all of the functions and the likeof the present invention described above by software. Alternatively, apart or all of the functions and the like of the present inventiondescribed above may be realized by using both hardware and software.

Further, the control lines and the information lines illustrated in thedrawings are illustrated as ones that are thought to be necessary forexplanation. All of the control lines and the information lines on aproduct are not necessarily illustrated in the drawings. In fact, it maybe thought that almost all components are mutually connected to eachother.

REFERENCE SINGS LIST

100 . . . imaging apparatus, 101 . . . main controller, 104 a . . .basic operation executing unit, 104 b . . . camera function executingunit, 110 a . . . basic operation program storage area, 110 b . . .camera function program storage area, 120 . . . video input unit, 123 .. . first video input unit, 123 b . . . first imaging device, 124 . . .second video input unit, 124 b . . . second imaging device, 165 . . .light quantity sensor, 224 . . . second video input unit, 224 b . . .second imaging device, 228 . . . resolution converter

1. An imaging apparatus comprising: a video input unit configured totake an image of an object to generate an image signal of the object; avideo signal processor configured to generate a taken image of theobject on a basis of the image signal; and a controller configured to:detect motion information of the object on the basis of the imagesignal; cause the video input unit to take an image of the object on abasis of the motion information multiple times so as to differentiate anexposure amount thereof; and cause the video signal processor togenerate an HDR synthetic image of the object on the basis of aplurality of image signals whose exposure amounts are different fromeach other.
 2. The imaging apparatus according to claim 1, wherein thevideo input unit includes: a first video input unit provided with afirst imaging device for first resolution; and a second video input unitprovided with a second imaging device for second resolution, resolutionof the second resolution being lower than that of the first resolution,wherein the controller is configured to: compare magnitude of the motioninformation with a first motion threshold value; cause the first videoinput unit to take an image of the object in succession so as todifferentiate an exposure amount thereof in a case where it isdetermined that the magnitude of the motion information is less than thefirst motion threshold value; and cause the video signal processor togenerate the HDR synthetic image with the first resolution, and wherein,in a case where it is determined that the magnitude of the motioninformation is equal to or more than the first motion threshold value,the controller is configured to: cause the first video input unit andthe second video input unit to take an image of the object at the sametime so as to differentiate exposure amounts thereof; cause the videosignal processor to convert resolution of the image signal generated bythe first video input unit into the second resolution; cause the videosignal processor to convert resolution of the image signal generated bythe second video input unit into the first resolution; and cause thevideo signal processor to generate the HDR synthetic image with thefirst resolution and the HDR synthetic image with the second resolution.3. The imaging apparatus according to claim 2, wherein the controller isconfigured to: compare the magnitude of the motion information with thefirst motion threshold value and a second motion threshold value, thesecond motion threshold value being larger than the first motionthreshold value; cause the first video input unit and the second videoinput unit to take an image of the object at the same time so as todifferentiate exposure amounts thereof in a case where it is determinedthat the magnitude of the motion information is equal to or more thanthe first motion threshold value and is equal to or less than the secondmotion threshold value; then cause the second video input unit to takean image of the object again so as to differentiate an exposure amountthereof; and cause the video signal processor to generate the HDRsynthetic image with the second resolution on the basis of the imagesignal generated by taking the image again.
 4. The imaging apparatusaccording to claim 1, further comprising: a light quantity sensorconfigured to measure light quantity of periphery, wherein thecontroller is configured to cause the video input unit to take an imageof the object multiple times so as to differentiate an exposure amountthereof on a basis of the motion information and the light quantity. 5.The imaging apparatus according to claim 4, wherein the video input unitincludes: a first video input unit provided with a first imaging devicefor first resolution; a second video input unit provided with a secondimaging device for the first resolution; and a resolution converterconfigured to convert resolution of each of the first imaging device andthe second imaging device, wherein the controller is configured to:compare magnitude of the motion information with a third motionthreshold value; cause the first video input unit to take an image ofthe object in succession so as to differentiate an exposure amountthereof in a case where it is determined that the magnitude of themotion information is less than the third motion threshold value; causethe video signal processor to generate the HDR synthetic image with thefirst resolution; compare the light quantity with a first light quantitythreshold value and a second light quantity threshold value, which islarger than the first light quantity threshold value, in a case where itis determined that the magnitude of the motion information is equal toor more than the third motion threshold value; cause the first videoinput unit and the second video input unit to take an image of theobject at the same time so as to differentiate exposure amounts thereofin a case where it is determined that the light quantity exceeds thesecond light quantity threshold value; cause the video signal processorto generate the HDR synthetic image with the first resolution; cause theresolution converter to convert the resolution of the second imagingdevice into second resolution in a case where it is determined that thelight quantity is equal to or more than the first light quantitythreshold value and is equal to or less than the second light quantitythreshold value; cause the first video input unit and the second videoinput unit to take an image of the object at the same time so as todifferentiate exposure amounts thereof; cause the resolution converterto convert resolution of the image signal generated by the first videoinput unit into the second resolution; cause the resolution converter toconvert resolution of the image signal generated by the second videoinput unit into the first resolution; cause the video signal processorto generate the HDR synthetic image with the first resolution and theHDR synthetic image with the second resolution; cause the resolutionconverter to convert resolution of each of the first imaging device andthe second imaging device into the second resolution in a case where itis determined that the light quantity is less than the first lightquantity threshold value; cause the first video input unit and thesecond video input unit to take an image of the object at the same timeso as to differentiate exposure amounts thereof; and cause the videosignal processor to generate the HDR synthetic image with the secondresolution.
 6. The imaging apparatus according to claim 5, wherein thecontroller is configured to: cause the resolution converter to convertresolution of each of the first imaging device and the second imagingdevice from the first resolution to the second resolution by grouping aplurality of pixels; and cause the resolution converter to convert theresolution of each of the first imaging device and the second imagingdevice from the second resolution to the first resolution by releasingof the grouping of the plurality of images.
 7. The imaging apparatusaccording to claim 1, wherein the controller is configured to detect themotion information of the object on a basis of a focal distance of thevideo input unit.
 8. The imaging apparatus according to claim 1, furthercomprising a display, wherein the display is configured to: display athumbnail image of the HDR synthetic image; and display, when thethumbnail image is selected, the HDR synthetic image corresponding tothe selected thumbnail image.
 9. An imaging method for an imagingapparatus, the imaging apparatus comprising a video input unit, a videosignal processor, and a controller, the imaging method comprising: avideo inputting step of taking an image of an object by the video inputunit to generate an image signal of the object; a motion informationdetecting step of detecting motion information of the object on a basisof the image signal by the controller; an imaging step of causing thevideo input unit to take an image of the object multiple times on abasis of the motion information so as to differentiate an exposureamount by the controller; and an HDR synthetic image generating step ofgenerating an HDR synthetic image of the object on the basis of aplurality of image signals whose exposure amounts are different fromeach other by the video signal processor.
 10. An imaging program for animaging apparatus, the imaging apparatus comprising a video input unit,a video signal processor, and a controller, the imaging program causingthe imaging apparatus to execute: a video inputting step of causing thevideo input unit to take an image of an object to generate an imagesignal of the object; a motion information detecting step of causing thecontroller to detect motion information of the object on a basis of theimage signal; an imaging step of causing the video input unit to take animage of the object multiple times so as to differentiate an exposureamount thereof on a basis of the motion information by the controller;and an HDR synthetic image generating step of causing the video signalprocessor to generate an HDR synthetic image of the object on the basisof a plurality of image signals whose exposure amounts are differentfrom each other.